Display apparatus and control method for display of a virtual flat screen

ABSTRACT

A display apparatus is disclosed. The display apparatus includes a sensor, a display, and a processor. The processor is configured to generate a virtual flat screen based on an angle of the display identified through the sensor, correct a size of a content based on a distance between the virtual flat screen and the display, project the corrected content to a virtual space, map the content projected to the virtual space to the virtual flat screen, render the virtual flat screen that the content is mapped to, and display the rendered virtual flat screen on the display.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based on and claims priority under 35 U.S.C. §119(a) of a Korean Patent Application No. 10-2020-0020285 filed on Feb.19, 2020 in the Korean Intellectual Property Office, the disclosure ofwhich is incorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to a display apparatus and a control methodthereof, and more particularly, to a foldable display apparatus and acontrol method thereof.

2. Description of Related Art

With the development of recent technologies, various electronicapparatuses are being developed. In particular, foldable displayapparatuses wherein the flexible displays are mounted are beingdeveloped.

A foldable display apparatus is a display apparatus which can be foldedor unfolded according to a user manipulation, and a user can use variousfunctions provided by a foldable display apparatus while folding thefoldable display apparatus by a specific angle. As an example, a usercan view or manipulate a content displayed on a foldable displayapparatus while folding the foldable display apparatus by a specificangle.

However, as a foldable display apparatus is folded by a specific angle,a content displayed on the foldable display apparatus may be viewed tobe curved to a user. In this case, a user viewing or manipulating thecontent may feel visually inconvenienced.

SUMMARY

This disclosure was devised for addressing the aforementioned problem,and one purpose of this disclosure is for providing a display apparatusthat can provide an uncurved screen to a user while it is folded by aspecific angle, and a control method thereof.

A display apparatus according to an embodiment of this disclosure mayinclude a sensor, a display, and a processor configured to generate avirtual flat screen based on an angle of the display identified throughthe sensor, correct a size of a content based on a distance between thevirtual flat screen and the display, project the corrected content to avirtual space, map the content projected to the virtual space to thevirtual flat screen, render the virtual flat screen to which the contentis mapped, and display the rendered virtual flat screen on the display.

The display apparatus may further include a camera, and the processormay identify a user's pupil in an image photographed through the camera,identify the direction of the user's gaze based on a location of theidentified user's pupil, rotate the virtual flat screen to which thecontent is mapped in a direction perpendicular to the direction of theuser's gaze, and render the rotated virtual flat screen and display therendered virtual flat screen on the display.

In addition, the processor may identify a rotating direction of thedisplay based on a plurality of pulse signals of different phasesreceived from the sensor, and identify the angle of the display based onthe rotating direction and the number of the pulse signals received fromthe sensor.

Further, the processor may, based on a coordinate of a first point ofthe display, identify coordinates of second to fourth points based onthe angle of the display, the horizontal length of the display, and thevertical length of the display, and generate the virtual flat screenbased on the coordinates of the first to fourth points.

The processor may also identify a distance from the virtual flat screento the display located in a vertical direction of the virtual flatscreen, and correct the size of the content based on information on theidentified distance and correction coefficients wherein differentcorrection coefficients are matched for each distance.

In addition, the processor may correct the content to be larger as thedistance value from the virtual flat screen to the display is larger.

Here, the virtual space to which the corrected content is projected maybe a three-dimensional space having a form of a curved surface, and theprocessor may map the content projected to the three-dimensional spacehaving a form of a curved surface to the virtual flat screen having aform of a two-dimensional flat surface based on a curved surface-flatsurface mapping algorithm.

Further, the processor may, based on a gaze vector corresponding to thedirection of the user's gaze, identify a first vector perpendicular tothe gaze vector, and based on the first vector and a second vectorcorresponding to the virtual flat screen, identify the angle between thefirst and second vectors, and rotate the virtual flat screen based onthe angle between the first and second vectors.

The processor may also, based on the angle between the first and secondvectors being an angle exceeding a predetermined angle in the directionwherein the user's pupil was identified based on the second vector,rotate the virtual flat screen in the direction wherein the user's pupilwas identified; and based on the angle between the first and secondvectors being an angle exceeding a predetermined angle in an oppositedirection to the direction wherein the user's pupil was identified basedon the second vector, rotate the virtual flat screen in the oppositiondirection to the direction wherein the user's pupil was identified.

Meanwhile, a display apparatus according to an embodiment of thedisclosure may include a sensor, a camera, a display, and a processorconfigured to, based on a direction of a user's gaze included in animage photographed through the camera, identify a gaze vector, dividethe display into a first display and a second display based on a linewherein the display is folded, generate a virtual flat screenperpendicular to the gaze vector based on one point of the first displayand the gaze vector, project a content to be displayed on the seconddisplay to the virtual flat screen based on the angle of the displayidentified through the sensor, render the virtual flat screen to whichthe content is projected, and display the rendered virtual flat screenon the display.

The processor may also identify a projection angle based on a differencebetween a predetermined angle and the angle of the display, and projectthe content to the virtual flat screen based on the projection angle.

In addition, the processor may project the content to the virtual flatscreen rotated by the projection angle based on an axis corresponding tothe folding line.

Meanwhile, a control method of a display apparatus according to anembodiment of the disclosure may include the steps of generating avirtual flat screen based on an angle of a display, correcting a size ofa content based on a distance between the virtual flat screen and thedisplay, projecting the corrected content to a virtual space, mappingthe content projected to the virtual space to the virtual flat screen,and rendering the virtual flat screen to which the content is mapped,and displaying the rendered virtual flat screen on the display.

The control method of a display apparatus may further include the stepof photographing a user, and in the step of displaying, the user's pupilmay be identified in the photographed image, the direction of the user'sgaze may be identified based on a location of the identified user'spupil, the virtual flat screen to which the content is mapped may berotated in a direction perpendicular to the direction of the user'sgaze, and the rotated virtual flat screen may be rendered and therendered virtual flat screen may be displayed on the display.

In addition, in the step of identifying the angle, a rotating directionof the display may be identified based on a plurality of pulse signalsof different phases received from the sensor; and the angle of thedisplay may be identified based on the rotating direction and the numberof the pulse signals received from the sensor.

Further, in the step of generating, based on a coordinate of a firstpoint of the display, coordinates of second to fourth points may beidentified based on the angle of the display, the horizontal length ofthe display, and the vertical length of the display. The virtual flatscreen may be generated based on the coordinates of the first to fourthpoints.

In the step of correcting, a distance from the virtual flat screen tothe display located in a vertical direction of the virtual flat screenmay be identified, and the size of the content may be corrected based oninformation on the identified distance and correction coefficients,wherein different correction coefficients are matched for each distance.

In addition, in the step of correcting, the content may be corrected tobe larger as the distance value from the virtual flat screen to thedisplay is larger.

Here, the virtual space to which the corrected content is projected maybe a three-dimensional space having a form of a curved surface. In thestep of mapping, the content projected to the three-dimensional spacehaving a form of a curved surface may be mapped to the virtual flatscreen having a form of a two-dimensional flat surface based on a curvedsurface-flat surface mapping algorithm.

Further, in the step of rotating, based on a gaze vector correspondingto the direction of the user's gaze, a first vector perpendicular to thegaze vector may be identified, and based on the first vector and asecond vector corresponding to the virtual flat screen, the anglebetween the first and second vectors may be identified, and the virtualflat screen may be rotated based on the angle between the first andsecond vectors.

According to the various embodiments of the disclosure as above, adisplay apparatus that can provide an uncurved screen to a user while itis folded by a specific angle, and a control method thereof can beprovided.

Before undertaking the DETAILED DESCRIPTION below, it may beadvantageous to set forth definitions of certain words and phrases usedthroughout this patent document: the terms “include” and “comprise,” aswell as derivatives thereof, mean inclusion without limitation; the term“or,” is inclusive, meaning and/or; the phrases “associated with” and“associated therewith,” as well as derivatives thereof, may mean toinclude, be included within, interconnect with, contain, be containedwithin, connect to or with, couple to or with, be communicable with,cooperate with, interleave, juxtapose, be proximate to, be bound to orwith, have, have a property of, or the like; and the term “controller”means any device, system or part thereof that controls at least oneoperation, such a device may be implemented in hardware, firmware orsoftware, or some combination of at least two of the same. It should benoted that the functionality associated with any particular controllermay be centralized or distributed, whether locally or remotely.

Moreover, various functions described below can be implemented orsupported by one or more computer programs, each of which is formed fromcomputer readable program code and embodied in a computer readablemedium. The terms “application” and “program” refer to one or morecomputer programs, software components, sets of instructions,procedures, functions, objects, classes, instances, related data, or aportion thereof adapted for implementation in a suitable computerreadable program code. The phrase “computer readable program code”includes any type of computer code, including source code, object code,and executable code. The phrase “computer readable medium” includes anytype of medium capable of being accessed by a computer, such as readonly memory (ROM), random access memory (RAM), a hard disk drive, acompact disc (CD), a digital video disc (DVD), or any other type ofmemory. A “non-transitory” computer readable medium excludes wired,wireless, optical, or other communication links that transporttransitory electrical or other signals. A non-transitory computerreadable medium includes media where data can be permanently stored andmedia where data can be stored and later overwritten, such as arewritable optical disc or an erasable memory device.

Definitions for certain words and phrases are provided throughout thispatent document, those of ordinary skill in the art should understandthat in many, if not most instances, such definitions apply to prior, aswell as future uses of such defined words and phrases.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present disclosure and itsadvantages, reference is now made to the following description taken inconjunction with the accompanying drawings, in which like referencenumerals represent like parts:

FIG. 1 illustrates a block diagram for illustrating a display apparatusaccording to an embodiment of the disclosure;

FIG. 2A illustrates a diagram illustrating a display apparatus in afolded state according to an embodiment of the disclosure;

FIG. 2B illustrates a diagram illustrating a side surface of a displayapparatus in a folded state according to an embodiment of thedisclosure;

FIG. 2C illustrates a diagram illustrating a side surface of a displayapparatus in a folded state according to an embodiment of thedisclosure;

FIG. 3 illustrates a diagram for illustrating information on correctioncoefficients according to an embodiment of the disclosure;

FIG. 4 illustrates a diagram for illustrating an embodiment ofprojecting a content based on a gaze vector according to an embodimentof the disclosure;

FIG. 5A illustrates a diagram for illustrating an embodiment of rotatinga virtual flat screen according to an embodiment of the disclosure;

FIG. 5B illustrates a diagram for illustrating an embodiment of notrotating a virtual flat screen according to an embodiment of thedisclosure;

FIG. 5C illustrates a diagram for illustrating an embodiment of rotatinga virtual flat screen according to an embodiment of the disclosure;

FIG. 6 illustrates a block diagram for illustrating a display apparatusaccording to an embodiment of the disclosure;

FIG. 7 illustrates a diagram for illustrating an operation of a displayapparatus according to an embodiment of the disclosure;

FIG. 8A illustrates a diagram for illustrating an embodiment ofprojecting a content to a virtual flat screen according to an embodimentof the disclosure;

FIG. 8B illustrates a diagram for illustrating an embodiment ofprojecting a content to a virtual flat screen according to an embodimentof the disclosure;

FIG. 9 illustrates a detailed block diagram for illustrating a displayapparatus according to an embodiment of the disclosure;

FIG. 10 illustrates a flow chart for illustrating an operation of adisplay apparatus according to an embodiment of the disclosure; and

FIG. 11 illustrates a flow chart for illustrating an operation of adisplay apparatus according to an embodiment of the disclosure.

DETAILED DESCRIPTION

FIGS. 1 through 11, discussed below, and the various embodiments used todescribe the principles of the present disclosure in this patentdocument are by way of illustration only and should not be construed inany way to limit the scope of the disclosure. Those skilled in the artwill understand that the principles of the present disclosure may beimplemented in any suitably arranged system or device.

Hereinafter, various embodiments of the disclosure will be describedwith reference to the accompanying drawings. However, it should be notedthat the various embodiments are not for limiting the technologydescribed in the disclosure to a specific embodiment, but they should beinterpreted to include various modifications, equivalents, and/oralternatives of the embodiments of the disclosure. Also, with respect tothe detailed description of the drawings, similar components may bedesignated by similar reference numerals.

In the disclosure, expressions such as “have,” “may have,” “include,”and “may include” should be construed as denoting that there are suchcharacteristics (e.g.: elements such as numerical values, functions,operations, and components), and the expressions are not intended toexclude the existence of additional characteristics.

Also, in the disclosure, the expressions “A or B,” “at least one of Aand/or B,” or “one or more of A and/or B” and the like may include allpossible combinations of the listed items. For example, “A or B,” “atleast one of A and B,” or “at least one of A or B” may refer to all thefollowing cases: (1) including at least one A, (2) including at leastone B, or (3) including at least one A and at least one B.

Further, the expressions “first,” “second,” and the like used in thedisclosure may be used to describe various elements regardless of anyorder and/or degree of importance. Also, such expressions are used onlyto distinguish one element from another element, and are not intended tolimit the elements.

Meanwhile, the description in the disclosure that one element (e.g.: afirst element) is “(operatively or communicatively) coupled with/to” or“connected to” another element (e.g.: a second element) should beinterpreted to include both the case where the one element is directlycoupled to the another element, and the case where the one element iscoupled to the another element through still another element (e.g.: athird element). In contrast, the description that one element (e.g.: afirst element) is “directly coupled” or “directly connected” to anotherelement (e.g.: a second element) can be interpreted to mean that stillanother element (e.g.: a third element) does not exist between the oneelement and the another element.

Also, the expression “configured to” used in the disclosure may beinterchangeably used with other expressions such as “suitable for,”“having the capacity to,” “designed to,” “adapted to,” “made to,” and“capable of,” depending on cases. Meanwhile, the term “configured to”does not necessarily mean that a device is “specifically designed to” interms of hardware. Instead, under some circumstances, the expression “adevice configured to” may mean that the device “is capable of”performing an operation together with another device or component. Forexample, the phrase “a processor configured to perform A, B, and C” maymean a dedicated processor (e.g.: an embedded processor) for performingthe corresponding operations, or a generic-purpose processor (e.g.: aCPU or an application processor) that can perform the correspondingoperations by executing one or more software programs stored in a memorydevice.

In addition, in the disclosure, “a module” or “a part” performs at leastone function or operation, and it may be implemented as hardware orsoftware, or as a combination of hardware and software. Further, aplurality of “modules” or “parts” may be integrated into at least onemodule and implemented as at least one processor (not shown), except“modules” or “parts” that need to be implemented as specific hardware.

Hereinafter, the disclosure will be described in detail with referenceto the accompanying drawings.

FIG. 1 illustrates a block diagram for illustrating a display apparatusaccording to an embodiment of the disclosure.

A display apparatus 100 according to an embodiment of the disclosure maybe a foldable display apparatus. In this example, the display apparatus100 may include a display 140 that can be folded (e.g., a flexibledisplay), and a hinge for folding the display 140 from the upper side tothe lower side, or from the lower side to the upper side. Here, thehinge is a component including a circular gear consisting of a pluralityof crews; and if a user applies force to the display 140 for folding thedisplay 140, the circular gear of the hinge rotates, and accordingly,the display 140 may be folded. As an example, if a user applies force tothe display 140 for folding the display 140, the circular gear mayrotate in a counter-clockwise direction, and accordingly, the display140 may be folded.

Referring to FIG. 1, the display apparatus 100 according to anembodiment of the disclosure may include a sensor 110, a camera 120, amemory 130, a display 140, and a processor 150. Note that this is aconfiguration according to an embodiment, and depending on variousembodiments according to this disclosure, the display apparatus 100 mayfurther include components other than the aforementioned components, orwith some of the components among the aforementioned components beingomitted.

The processor 150 may identify the angle of the display 140 by loading adisplay angle measurement module 151 stored in the memory 130.

Specifically, the processor 150 may identify the angle of the display140 based on sensing data received from the sensor 110.

Here, the sensor 110 is a component outputting pulse signals in case thecircular gear included in the hinge rotates, and it may be implementedas an encoder.

Specifically, the sensor 110 may be implemented as an encoder includinga plurality of light emitting diodes (e.g., LEDs), a rotation discincluding a plurality of slots, and a plurality of light receivingdiodes (e.g., photodiodes), and it may be coupled to the circular gearof the hinge. Also, as the circular gear of the hinge rotates, if lightsoutput by the plurality of light emitting diodes pass through the slotsof the rotation disc and reach the plurality of light receiving diodes,the sensor 110 may output a plurality of pulse signals.

Next, the processor 150 may identify the angle of the display 140 basedon the number of the pulse signals received from the sensor 110.Specifically, the processor 150 may identify the angle which is a valueof multiplying the number of the pulse signals received from the sensor110 with a predetermined angle as the angle of the display 140. As anexample, in a case in which a predetermined angle is a°, if n pulsesignals are received from the sensor 110, the processor 150 may identifythe angle of the display 140 as (a×n)°. For this, the memory 130 maystore information on the predetermined angle.

Meanwhile, based on the plurality of pulse signals of different phasesreceived from the sensor 110, the processor 150 may identify therotating direction of the circular gear. As described above, the sensor110 may output lights through the plurality of light emitting diodes. Inthis case, a light output by a first light emitting diode among theplurality of light emitting diodes passes through the slot of therotation disc and reaches a first light receiving diode, the sensor 110may output a pulse signal of an A phase. In a case in which a lightoutput by a second light emitting diode among the plurality of lightemitting diodes passes through the slot of the rotation disc and reachesa second light receiving diode, the sensor 110 may output a pulse signalof a B phase. For this example, the first light emitting diode and thesecond light emitting diode of the sensor 110 may be arranged indifferent locations and output lights toward the rotation disc, and thefirst light receiving diode may be arranged in a location wherein alight output by the first light emitting diode can be received, and thesecond light receiving diode may be arranged in a location wherein alight output by the second light emitting diode can be received.

Next, based on the phase difference between the pulse signal of the Aphase and the pulse signal of the B phase, the processor 150 mayidentify the rotating direction of the circular gear. Specifically, ifthe phase of the pulse signal of the A phase is higher than the phase ofthe pulse signal of the B phase by 90°, the processor 150 may identifythat the circular gear rotates in a clockwise direction; and if thephase of the pulse signal of the B phase is higher than the phase of thepulse signal of the A phase by 90°, the processor 150 may identify thatthe circular gear rotates in a counter-clockwise direction.

Accordingly, the processor 150 may identify the angle of the display 140in further consideration of the rotating direction of the circular gear.Specifically, the processor 150 may identify the rotating direction ofthe circular gear based on the phase difference between the pulse signalof the A phase and the pulse signal of the B phase received from thesensor 110; and if the circular gear rotates in a counter-clockwisedirection, the processor 150 may identify an angle which is a value ofmultiplying the number of the pulse signals received from the sensor 110with a predetermined angle as the angle of the display 140; and if thecircular gear rotates in a counter-clockwise direction and then rotatesin a clockwise direction, the processor 150 may identify the differencebetween an angle which is a value of multiplying the number of the pulsesignals received until the phase of the pulse signal of the A phasebecomes higher than the phase of the pulse signal of the B phase by 90°with a predetermined angle and an angle which is a value of multiplyingthe number of the pulse signals received after the phase of the pulsesignal of the B phase becomes higher than the phase of the pulse signalof the A phase by 90° as the angle of the display 140.

When the angle of the display 140 is identified, the processor 150 mayload the rendering module 153 stored in the memory 130, and therebyrender a content.

Specifically, the processor 150 may correct a size of a content based ona distance from the display 140 to a virtual flat screen, project thecontent having the corrected size to a virtual space P, map the contentprojected to the virtual space P to the virtual flat screen, and renderthe content mapped to the virtual flat screen and display the content onthe display 140.

For this, the processor 150 may first generate a virtual flat screenbased on the angle of the display 140. Specifically, the processor 150may generate a virtual flat screen based on the angle of the display 140and the horizontal and vertical lengths of the display 140. Explanationin this regard will be made with reference to FIG. 2A.

Referring to FIG. 2A, based on the angle θ of the display 140, thehorizontal length a cm (hereinafter, the unit will be omitted) of thedisplay 140 and the vertical length b of the display 140 (here, b=b1+b2,and b1 may be the vertical length of the display 140 located in theupper part based on the folding line when the display 140 is folded, andb2 may be the vertical length of the display 140 located in the lowerpart based on the folding line when the display 140 is folded), theprocessor 150 may set the coordinates of the four points 141, 142, 143,144 including the reference point 141.

Specifically, the processor 150 may set the coordinate of the point 1(141) which is the reference point 141 as (0, 0), set the coordinate ofthe point 2 (142) as the coordinate (a, 0) based on the horizontallength of the display 140, set the coordinate of the point 3 (143) as(0, b-k) based on the vertical length of the display 140 and the angleof the display 140, and set the coordinate of the point 4 (144) as (a,b-k) based on the horizontal length and the vertical length of thedisplay 140 and the angle of the display 140.

Here, the coordinate b-k of the y axis of the points 3, 4 (143, 144) maybe calculated through triangulation. Explanation in this regard will bemade with reference to FIG. 2B.

Referring to FIG. 2B, when the display apparatus 100 in a folded stateis viewed from the side surface, the vertical length of the virtual flatscreen 145 may be c. Also, the angle between the display 140 located inthe upper part based on the folding line and the virtual flat screen 145may be α, and the angle between the display 140 located in the lowerpart based on the folding line and the virtual flat screen 145 may be β.

In this case, through triangulation, the processor 150 may acquire aformula such as b1/sin β=b2/sin α=c/sin θ (here, b1 may be the verticallength of the display 140 located in the upper part based on the foldingline when the display 140 is folded, b2 may be the vertical length ofthe display 140 located in the lower part based on the folding line whenthe display 140 is folded, θ may be the angle of the display 140, c maybe the vertical length of the virtual flat screen 145, a may be theangle between the display 140 located in the upper part based on thefolding line and the virtual flat screen 145, and β may be the anglebetween the display 140 located in the lower part based on the foldingline when the display 140 is folded and the virtual flat screen 145).

Next, based on the formula 1: b1 sin α=b2 sin β, formula 2: b2 sin θ=csin α, and formula 3: b1 sin θ=c sin β, and the angle θ of the display140, the vertical length b1 of the display 140 located in the upper partbased on the folding line, and the vertical length b2 of the display 140located in the lower part based on the folding line, the processor 150may calculate the vertical length c of the virtual flat screen 145, theangle α between the display 140 located in the upper part based on thefolding line and the virtual flat surface, and the angle β between thedisplay 140 located in the lower part based on the folding line and thevirtual flat surface.

Next, the processor 150 may calculate the vertical length of the virtualflat screen 145 by summing up b1 cos α and b2 cos β. Here, the value ofsumming up b1 cos α and b2 cos β becomes the aforementioned coordinateb-k of the y axis of the points 3, 4 (143, 144).

If the virtual flat screen 145 is generated based on the points 1, 2, 3,4 (141, 142, 143, 144) (i.e., if the virtual flat screen 145 wherein thepoint 1 and the point 2, the point 2 and the point 3, the point 3 andthe point 4, and the point 4 and the point 1 are connected isgenerated), the processor 150 may project a content to the virtual spaceP based on the distance between the virtual flat screen 145 and thedisplay 140.

For this, the processor 150 may identify the distance between thevirtual flat screen 145 and the display 140. Explanation in this regardwill be made with reference to FIG. 2C.

Referring to FIG. 2C, the processor 150 may calculate a distance H fromthe virtual flat screen 145 to the display 140 located in the verticaldirection of the virtual flat screen 145.

For this, through triangulation, the processor 150 may acquire a formulasuch as b1/sin 90°=h/sin α=c1/sin θ1 (here, b1 may be the verticallength of the display 140 located in the upper part based on the foldingline when the display 140 is folded, h may be the vertical distance fromthe virtual flat screen 145 to the display 140, θ1 may be the anglebetween a virtual flat surface perpendicular to the virtual flat screen145 and the display 140 located in the upper part based on the foldingline, a may be the angle between the display 140 located in the upperpart based on the folding line and the virtual flat surface, and c1 maybe the distance from the point that descended as much as the verticaldistance h in the virtual flat screen 145 to the point located on theuppermost end in the virtual flat screen 145 (i.e., the point whereinthe coordinate of the y axis is the aforementioned b-k).

Next, based on the formula 1: b1 sin α=h sin 90°, formula 2: h sin θ1=c1sin α, and formula 3: b1 sin θ1=c1 sin 90°, and the vertical length b1of the display 140 located in the upper part based on the folding line,and the angle α between the display 140 located in the upper part basedon the folding line and the virtual flat surface, the processor 150 maycalculate the vertical distance h between the virtual flat screen 145and the display 140.

Also, through triangulation, the processor 150 may acquire a formulasuch as b2/sin 90°=h/sin β=c2/sin θ2 (here, b2 may be the verticallength of the display 140 located in the lower part based on the foldingline, h may be the vertical distance from the virtual flat screen 145 tothe display 140, θ2 may be the angle between the virtual flat surfaceperpendicular to the virtual flat screen 145 and the display 140 locatedin the lower part based on the folding line, β may be the angle betweenthe display 140 located in the lower part based on the folding line andthe virtual flat surface, and c2 may be the distance from the point thatdescended as much as the vertical distance h in the virtual flat screen145 to the point located on the lowermost end in the virtual flat screen145).

Next, based on the formula 1: b2 sin β=h sin 90°, formula 2: h sin θ2=c2sin β, and formula 3: b2 sin θ2=c2 sin 90°, and the vertical length b2of the display 140 located in the lower part based on the folding line,and the angle β between the display 140 located in the lower part basedon the folding line and the virtual flat surface, the processor 150 maycalculate the vertical distance h between the virtual flat screen 145and the display 140.

In FIG. 2C, one length h is illustrated, but the display 140 may includea plurality of pixels; and the processor 150 may calculate a pluralityof distances from the virtual flat screen 145 to the plurality of pixelslocated in a vertical direction of the virtual flat screen 145 based onthe aforementioned triangular measurement method.

Next, the processor 150 may project a content to a virtual space P basedon the distance between the virtual flat screen 145 and the display 140.

Specifically, the processor 150 may correct a size of a content based oninformation on correction coefficients wherein different correctioncoefficients are matched for each distance between the virtual flatscreen 145 and the display 140, and project the content having thecorrected size to the virtual space P.

As an example, the memory 130 according to an embodiment of thedisclosure may store information on correction coefficients (i.e.,correction coefficients LUT) as in FIG. 3.

In this case, if the vertical distance from the virtual flat screen 145to the first pixel of the display 140 is h1, the processor 150 maycorrect the size of a content corresponding to the first pixel by Q1times, and project the content having the size corrected by Q1 times tothe virtual space P. In a similar manner thereto, if the distance fromthe virtual flat screen 145 to the second pixel of the display 140 ish2, the processor 150 may correct the size of a content corresponding tothe second pixel by Q2 times, and project the content having the sizecorrected by Q2 times to the virtual space P, and if the distance fromthe virtual flat screen 145 to the nth pixel of the display 140 is hn,the processor 150 may correct the size of a content corresponding to thenth pixel by Qn times, and project the content having the size correctedby Qn times to the virtual space P. That is, the processor 150 maycorrect the size of a content corresponding to each of the first to nthpixels based on a correction coefficient Q, and project the contenthaving the corrected size to the virtual space P.

Meanwhile, in FIG. 3, the correction coefficient Q is bigger than 1, andmay have a bigger size as the vertical distance h becomes bigger.Accordingly, the processor 150 may correct a content to be bigger as thevertical distance h is bigger.

Also, as the correction coefficient Q is bigger than 1 and has a biggersize as the vertical distance h becomes bigger, the shape of the virtualspace P to which a corrected content is projected may be a curvedsurface. That is, the virtual space P may be a three-dimensional spacehaving a shape of a curved surface.

Accordingly, the processor 150 may perform a job of mapping a contentprojected to the virtual space P to the virtual flat screen 145. Thatis, the processor 150 may map a content projected to a three-dimensionalspace having a shape of a curved surface to the virtual flat screen 145having a shape of a two-dimensional flat surface. Specifically, theprocessor 150 may map a content projected to the virtual space P to thevirtual flat screen 145 based on a curved surface-flat surface mappingalgorithm. Here, the curved surface-flat surface mapping algorithm is analgorithm for converting a three-dimensional curved surface to atwo-dimensional flat surface, and it may be a parameterization polygonalmesh algorithm. In this case, the processor 150 may divide a contentprojected to the virtual space P into a plurality of areas through apolygonal mesh, and map the content divided into a plurality of areas todifferent areas of the virtual flat screen 145.

Accordingly, the processor 150 may acquire the virtual flat screen 145to which a content having a size corrected based on the aforementionedvertical distance h and correction coefficient Q is mapped.

Next, the processor 150 may render the virtual flat screen 145 to whicha content having a corrected size is mapped to display the screen on thedisplay 140, and display the rendered virtual flat screen 145 on thedisplay 140.

As described above, in the disclosure, a content is corrected todifferent sizes according to a folding angle of the display 140, and thevirtual flat screen 145 to which the content in the corrected size ismapped is rendered and displayed on the display 140, and accordingly, auser may be provided with a content in an uncurved form even in a statewherein the display 140 is folded.

Meanwhile, the processor 150 may render a content in furtherconsideration of a direction of a user's gaze. Explanation in thisregard will be made with reference to FIG. 4.

The processor 150 may rotate the virtual flat screen 145 based on adirection of a user's gaze, render the rotated virtual flat screen 145′to display the screen on the display 140, and display the renderedvirtual flat screen 145′ on the display 140.

For this, the processor 150 may identify a direction of a user's gaze byloading a gaze direction detection module 152 stored in the memory 130.

Specifically, the processor 150 may identify a user's pupil in an imagephotographed through the camera 120 through an object detectionalgorithm, and identify the direction of the user's gaze based on thelocation of the identified user's pupil.

Here, the direction of the user's gaze may be expressed with a gazevector. Specifically, the direction of the user's gaze is a directionwhich a virtual line connecting from the center point of the eyeball tothe center point of the pupil is toward, and the processor 150 maycalculate a point which proceeded toward the inside of the eyeball fromthe point wherein the user's pupil is located as much as the radius ofthe eyeball as the center point of the eyeball, and acquire a virtualline connecting from the center point of the eyeball to the center pointof the user's pupil as the gaze vector of the user. For this, theprocessor 150 may store information on the radius of the eyeball (e.g.,12 mm).

Next, the processor 150 may acquire a vector which is perpendicular tothe user's gaze vector through a dot product of vectors. Specifically,if the user's gaze vector acquired based on the direction of the user'sgaze is V1=(X1, Y1, Z1), and a vector perpendicular to the user's gazevector is V2=(X2, Y2, Z2), through a formula: V1 dot V2=|V1∥V2| cos θ3=0(here, θ3 is the angle between the user's gaze vector V1 and the vectorperpendicular to the user's gaze vector V2, and the gaze vector V1 andthe vertical vector V2 is in a vertical relation, and thus V1 dot V2=0),the processor 150 may acquire the vector V2 perpendicular to the user'sgaze vector V1 (here, V2=(Y1, −X1, 0)).

Next, the processor 150 may identify an angle between a vectorcorresponding to the virtual flat screen 145 and the vectorperpendicular to the user's gaze vector. Here, the vector correspondingto the virtual flat screen 145 may be acquired based on the referencepoint 141 and at least one point on the virtual flat screen 145.Specifically, based on the reference point (0, 0, 0) and one point onthe virtual flat screen 145 (X3, Y3, Z3) (here, the coordinate of theone point may be determined as an optional one point based on thehorizontal and vertical lengths of the display 140 and the angle of thedisplay 140), the processor 150 may acquire a vector V3 corresponding tothe virtual flat screen 145 (here, V3=(X3, Y3, Z3)), and identify theangle between the vector corresponding to the virtual flat screen 145and the vector perpendicular to the user's gaze vector through a dotproduct of vectors. That is, through a dot product of the vector V2perpendicular to the user's gaze vector V1 (here, V2=(Y1, −X1, 0)) andthe vector V3 corresponding to the virtual flat screen 145 (here,V3=(X3, Y3, Z3)), the processor 150 may acquire a formula: θ4=cos⁻¹ (V2dot V3/(|V2∥V3|)) (here, θ4 is an angle between the vector V2perpendicular to the user's gaze vector V1 and the vector V3corresponding to the virtual flat screen 145), and identify θ4 throughthe aforementioned formula operation.

Next, based on the angle θ between the vector V2 perpendicular to theuser's gaze vector V1 and the vector V3 corresponding to the virtualflat screen 145, the processor 150 may rotate the virtual flat screen145 in a direction perpendicular to the direction of the user's gaze. Asan example, referring to FIG. 4, the processor 150 may rotate thevirtual flat screen 145 by the aforementioned angle θ4 based on thehorizontal axis of the display 140 including the reference point, andthereby acquire the virtual flat screen 145′ perpendicular to thedirection of the user's gaze.

Next, the processor 150 may render the virtual flat screen 145′ rotatedin a direction perpendicular to the direction of the user's gaze, anddisplay the rendered virtual flat screen 145′ on the display 140.

Accordingly, the user may be provided with a content in an uncurved formeven in a state wherein the display 140 is folded.

Meanwhile, in the above, explanation was made based on a case whereinthe display 140 is folded from the upper side to the lower side, or fromthe lower side to the upper side, but the technical idea of thedisclosure can be deemed to be also applied to a case wherein thedisplay 140 is folded from the left side to the right side, or from theright side to the left side.

FIG. 5A to FIG. 5C illustrate diagrams for example embodiments wherein adisplay apparatus according to an embodiment of the disclosure providesa content in an uncurved form. Hereinafter, for the convenience ofexplanation, explanation will be made based on the assumption that theangle between the vector V2 perpendicular to the user's gaze vector V1and the vector V3 corresponding to the virtual flat screen 145 is θ4.

As described above, the processor 150 may rotate the virtual flat screen145 in a direction perpendicular to the gaze vector. Specifically, if itis identified that the aforementioned angle θ4 is an angle exceeding apredetermined angle (e.g., 0 degree) in the direction wherein the user'spupil was identified based on the vector V3, the processor 150 mayrotate the virtual flat screen 145 by the angle θ in the directionwherein the user's pupil was identified; and if it is identified thatthe angle θ4 is an angle exceeding a predetermined angle (e.g., 0degree) in an opposite direction to the direction wherein the user'spupil was identified based on the vector V3, the processor 150 mayrotate the virtual flat screen 145 by the angle θ in an oppositedirection to the direction wherein the user's pupil was identified.

As an example, as illustrated in FIG. 4, if the angle θ is an angleexceeding 0 degree in the direction wherein the user's pupil wasidentified based on the vector V3, the processor 150 may rotate thevirtual flat screen 145 by the aforementioned angle θ4 in the directionwherein the user's pupil was identified based on the horizontal axis ofthe display 140 including the reference point, and thereby acquire thevirtual flat screen 145′ perpendicular to the direction of the user'sgaze. Next, the processor 150 may render the virtual flat screen 145′rotated in a direction perpendicular to the direction of the user'sgaze, and display the rendered virtual flat screen 145′ on the display140.

Alternatively, as illustrated in FIG. 5A, if the angle θ is an angleexceeding 0 degree in an opposite direction to the direction wherein theuser's pupil was identified based on the vector V3, the processor 150may rotate the virtual flat screen 145 by the aforementioned angle θ4 inan opposite direction to the direction wherein the user's pupil wasidentified based on the horizontal axis of the display 140 including thereference point, and thereby acquire the virtual flat screen 145′perpendicular to the direction of the user's gaze. Next, the processor150 may render the virtual flat screen 145′ rotated in a directionperpendicular to the direction of the user's gaze, and display therendered virtual flat screen 145′ on the display 140.

Alternatively, if the angle θ4 is a predetermined angle (e.g., 0degree), as illustrated in FIG. 5B, the processor 150 may not rotate thevirtual flat screen 145, and render the virtual flat screen 145 anddisplay the screen on the display 140.

Accordingly, no matter by which angle the display 140 is folded, theuser may be provided with a content in an uncurved form.

Note that the aforementioned example is merely an embodiment, anddepending on embodiments, if the aforementioned angle θ4 is an angleexceeding a predetermined angle (e.g., 0 degree) in the directionwherein the user's pupil was identified based on the vector V3, theprocessor 150 may rotate the virtual flat screen 145 by the angle θ inthe direction wherein the user's pupil was identified; and if the angleθ4 is an angle exceeding a predetermined angle (e.g., 0 degree) in anopposite direction to the direction wherein the user's pupil wasidentified based on the vector V3, the processor 150 may rotate thevirtual flat screen 145 by the angle θ in an opposite direction to thedirection wherein the user's pupil was identified.

Note that the aforementioned embodiment is an embodiment for a case inwhich the angle of the display 140 is changed while there is no changeto the location of the display apparatus, and the display apparatus 100according to an embodiment of the disclosure may be provided with acontent in an uncurved form if the location of the display apparatus ischanged from the first location to the second location while there is nochange to the angle of the display 140.

As an example, if the display apparatus 100 folded by a specific anglein the first location as illustrated in FIG. 5B is moved to the secondlocation as illustrated in FIG. 5C, the processor 150 may identify theangle θ4 between the vector V2 perpendicular to the user's gaze vectorV1 and the vector V3 corresponding to the virtual flat screen 145 in thesecond location. Next, if it is identified that the angle θ4 is an angleexceeding a predetermined angle (e.g., 0 degree) in the directionwherein the user's pupil was identified based on the vector V3, theprocessor 150 may rotate the virtual flat screen 145 by theaforementioned angle θ4 in the direction wherein the user's pupil wasidentified based on the horizontal axis of the display 140 including thereference point, and thereby acquire the virtual flat screen 145′perpendicular to the direction of the user's gaze. Next, the processor150 may render the virtual flat screen 145′ rotated in a directionperpendicular to the direction of the user's gaze, and display therendered virtual flat screen 145′ on the display 140. Alternatively, ifit is identified that the angle θ4 is an angle exceeding a predeterminedangle (e.g., 0 degree) in an opposite direction to the direction whereinthe user's pupil was identified based on the vector V3, the processor150 may rotate the virtual flat screen 145 by the aforementioned angleθ4 in an opposite direction to the direction wherein the user's pupilwas identified based on the horizontal axis of the display 140 includingthe reference point, and thereby acquire the virtual flat screen 145′perpendicular to the direction of the user's gaze. Next, the processor150 may render the virtual flat screen 145′ rotated in a directionperpendicular to the direction of the user's gaze, and display therendered virtual flat screen 145′ on the display 140.

Meanwhile, in the above, an embodiment of displaying a screen based on acontent of which size was corrected based on a correction coefficientwas explained. However, this is merely an embodiment, and the displayapparatus 100 according to an embodiment of the disclosure may provide acontent in an uncurved form without correcting the size of a content.Hereinafter, explanation in this regard will be made with reference toFIG. 6 and FIG. 7.

FIG. 6 illustrates a block diagram for illustrating a display apparatusaccording to an embodiment of the disclosure, and FIG. 7 is a diagramfor illustrating an operation of a display apparatus according to anembodiment of the disclosure. Hereinafter, explanation will be madewhile parts overlapping with the aforementioned explanation are omittedor abridged.

As described above, the display apparatus 100 according to an embodimentof the disclosure may include a sensor 110, a camera 120, a memory 130,a display 140, and a processor 150. Also, the processor 150 may identifyan angle of the display 140 by loading a display angle measurementmodule 151 stored in the memory 130, and identify a user's pupil byloading a gaze direction detection module 152 stored in the memory 130.Next, the processor 150 may identify a gaze vector corresponding to thedirection of the user's gaze based on the location of the identifiedpupil.

Referring to FIG. 7, the processor 150 may divide the display 140 into afirst display 710 and a second display 720 (or, divide into a firstdisplay area and a second display area) based on the folding line of thedisplay 140. Specifically, with the line wherein the display is foldedaccording to rotation of a hinge as the reference axis, the processor150 may identify a display 710 located in the upper part of thereference axis and a display 720 located in the lower part of thereference axis.

Next, the processor 150 may generate a virtual flat surfaceperpendicular to the gaze vector based on one point of the display 710in the upper part and the gaze vector. Specifically, the processor 150may generate a virtual flat surface perpendicular to the gaze vectorbased on one pixel among a plurality of pixels included in the display710 in the upper part and the gaze vector. As an example, in a case inwhich one point of the display 710 in the upper part is CP 1 (x1, y1,z1) as illustrated in FIG. 7, and the gaze vector is (a, b, c), theprocessor 150 may generate a virtual flat screen 730 expressed by a flatsurface equation such as a(x−x1)+b(y−y1)+c(z−z1)=0.

Next, the processor 150 may project a content to be displayed on thedisplay 720 in the lower part to the virtual flat screen 730 based onthe angle of the display 140.

For this, the processor 150 may identify the angle θ between the display720 in the lower part and the virtual flat screen 730. Specifically, theprocessor 150 may identify a difference between an angle when thedisplay 140 is in a flat state and an angle when the display 140 is in afolded state as the angle between the display 720 in the lower part andthe virtual flat screen 730. As an example, in a case in which the angleof the display 140 is α, the processor 150 may identify (180−α) degreeas the angle θ between the display 720 in the lower part and the virtualflat screen 730.

Next, the processor 150 may project a content to be displayed on thedisplay 720 in the lower part to the virtual flat screen 730 based on arotation matrix. Here, the rotation matrix may be expressed as below.

$\begin{pmatrix}x^{\prime} \\y^{\prime} \\z^{\prime}\end{pmatrix} = {\begin{pmatrix}1 & 0 & 0 \\0 & {\cos(\theta)} & {- {\sin(\theta)}} \\0 & {\sin(\theta)} & {\cos(\theta)}\end{pmatrix}\begin{pmatrix}x \\y \\z\end{pmatrix}}$

(Here, (x′, y′, z′) is one point of the virtual flat screen 730, (x, y,z) is one point of the display 720 in the lower part, and θ is the anglebetween the display 720 in the lower part and the virtual flat screen730.)

As an example, as illustrated in FIG. 7, if one point of the display 720in the lower part is CP 2 (x2, y2, z2), the processor 150 may project acontent to be displayed on the CP 2 to the CP 3 (x3, y3, z3) point whichis a point identified based on the aforementioned rotation matrix amongthe entire areas of the virtual flat screen 730. That is, if the CP 2point is one pixel, the processor 150 may identify a pixel valuecorresponding to a content to be displayed on the CP 2 point, andproject the identified pixel value to the CP 3 point among the entireareas of the virtual flat screen 730; and in a case in which the CP 2point is a block including a plurality of pixels, the processor 150 mayidentify a plurality of pixel values corresponding to a content to bedisplayed on the CP 2 point, and project the plurality of identifiedpixel values to the CP 3 point among the entire areas of the virtualflat screen 730.

As an example, as illustrated in FIG. 8A, in a case in which the anglebetween the display 720 in the lower part and the virtual flat screen730 is θ1, the processor 150 may project a content to be displayed onthe display 720 in the lower part to the virtual flat screen 730generated in a location rotated by the angle θ1 with the folding line asthe reference axis. As illustrated in FIG. 8B, in a case in which theangle between the display 720 in the lower part and the virtual flatscreen 730 is θ2, the processor 150 may project a content to bedisplayed on the display 720 in the lower part to the virtual flatscreen 730 generated in a location rotated by the angle θ2 with thefolding line as the reference axis.

In FIG. 7, it was illustrated that a content to be displayed on onepoint CP 2 of the display 720 in the lower part is projected to thevirtual flat screen 730, but it can be deemed that the processor 150 mayproject contents to be displayed on each of the plurality of points ofthe display 720 in the lower part to each of the plurality of points ofthe virtual flat screen 730.

Next, the processor 150 may render the virtual flat screen 730, whereincontents are projected to each of the plurality of points, and displaythe rendered virtual flat screen 730 on the display 140. That is, as inFIG. 8A and FIG. 8B, in a case in which a content to be displayed on thedisplay 720 in the lower part is projected to the virtual flat screen730, the processor 150 may render the virtual flat screen 730 to whichthe content is projected, and display the rendered virtual flat screen730 on the display 720 in the lower part.

Accordingly, the user may be provided with a content in an uncurved formeven in a state wherein the display 140 is folded.

Meanwhile, in the above, an embodiment of projecting a content to bedisplayed on the display 720 in the lower part is projected to thevirtual flat screen 730 was explained, but this is merely an embodiment,and it can be deemed that the aforementioned technical idea can beapplied to a case wherein a content to be displayed on the display 710in the upper part is projected to the virtual flat screen.

FIG. 9 illustrates a detailed block diagram for illustrating a displayapparatus according to an embodiment of the disclosure.

Referring to FIG. 9, a display apparatus according to an embodiment ofthe disclosure may include a sensor 110, a camera 120, a memory 130, adisplay 140, a communicator 160, an inputter 170, and a processor 150.Hereinafter, explanation will be made while parts overlapping with theaforementioned explanation are omitted or abridged.

The sensor 110 is a component for measuring the angle of the display140. As an example, the sensor 110 may be implemented as an encoder asdescribed above. However, this is merely an embodiment, and the sensor110 may be implemented as various types such as a potentiometer thatmeasures the angle of the display 140 based on a resistance value thatchanges according to the rotating angle of the display 140, a synchrosensor that measures the angle of the display 140 based on the change ofthe size of the voltage induced to the coil and the phase valueaccording to the rotating angle of the display 140, etc.

The camera 120 may generate an image by photographing a subject. Forexample, the camera 120 may photograph a user in the front side of thedisplay apparatus 100, and generate an image including the user.Accordingly, the processor 150 may identify the user's pupil byanalyzing the image photographed by the camera 120, and identify theuser's gaze vector based on the location of the identified user's pupil.Here, the gaze vector may not only be a gaze vector corresponding to theleft eye of the user, but also a gaze vector corresponding to the righteye.

The camera 120 as described above may be arranged on the panel on thecenter upper side of the display apparatus 100, but the disclosure isnot necessarily limited thereto, and the camera 120 may be arranged invarious locations such as the panel on the center lower side, the panelon the center left side, or the panel on the center right side, etc. ofthe display apparatus 100.

Meanwhile, the image photographed by the camera 120 may be stored in thememory 130.

The memory 130 may store an operating system (OS) for controlling theoverall operations of the components of the display apparatus 100 andinstructions or data related to the components of the display apparatus100.

Accordingly, the processor 150 may control a plurality of hardware orsoftware components of the display apparatus 100 by using variousinstructions or data stored in the memory 130, and load instructions ordata received from at least one among the other components on a volatilememory and process them, and store various data in a non-volatilememory.

In particular, the memory 130 may store information on correctioncoefficients. Accordingly, the processor 150 may correct the size of acontent based on information on correction coefficients whereindifferent correction coefficients are matched for each distance betweenthe virtual flat screen and the display 140, and project the contenthaving the corrected size to a virtual space.

Also, the memory 130 may store information on an object recognitionalgorithm or an artificial intelligence model that can identify anobject (e.g., a pupil) included in an image. Here, an artificialintelligence model may be a convolutional neural network (CNN) modelincluding a convolutional layer extracting characteristic information ofan image and a fully connected layer trained to identify an objectincluded in an image based on the extracted characteristic information,but is not necessarily limited thereto.

The display 140 may display various images. Here, an image is a conceptincluding at least one of a still image or a moving image, and thedisplay 140 may display various images such as a multimedia content, agame content, etc. Also, the display 140 may display various kinds ofuser interfaces (UIs) and icons.

In particular, the display 140 may display a content projected to avirtual flat screen generated in a direction perpendicular to thedirection of a user's gaze.

The display 140 as described above may be implemented as displays invarious forms such as a liquid crystal display (LCD) panel, lightemitting diodes (LED), organic light emitting diodes (OLED), LiquidCrystal on Silicon (LCoS), Digital Light Processing (DSP), etc. Also, inthe display 140, a driving circuit that may be implemented in forms suchas an a-si TFT, a low temperature poly silicon (LTPS) TFT, an organicTFT (OTFT), etc., a backlight unit, and the like may also be included.

The communicator 160 may perform communication with various externalapparatuses, and transmit and receive various data. As an example, thecommunicator 160 may be communicatively connected with externalapparatuses through various communication methods such as Wi-Fi,Bluetooth, etc., and transmit and receive various data to and fromexternal apparatuses. For this, the communicator 160 may include a Wi-Fimodule, a Bluetooth module, and a mobile communication module.

The inputter 170 may receive inputs of various user instructions. Theprocessor 150 may execute a function corresponding to a user instructioninput through the inputter 170.

As an example, the inputter 170 may receive a user input for folding,and the processor 150 may fold the display 140 based on the user input.Here, the user input for folding may not only be a user input of pushinga button provided on the display apparatus 100, but also a user input oftouching a UI for folding displayed on the display 140.

Also, the inputter 170 may receive a user instruction for projecting acontent. In this case, the processor 150 may project a content to avirtual flat screen based on the user instruction, render the virtualflat screen, and display the screen on the display 140.

For this, the inputter 170 may be implemented as an input panel. Aninput panel may be implemented as a touch pad type, a keypad typeincluding various kinds of function keys, number keys, special keys,character keys, etc., or a touch screen type.

The microphone (not shown) may receive a user voice. Here, a user voicemay be a voice for executing a specific function of the displayapparatus 100. When a user voice is received through the microphone (notshown), the processor 150 may analyze the user voice through a speech totext (STT) algorithm, and perform a function corresponding to the uservoice.

As an example, if a user voice for projecting a content is receivedthrough the microphone (not shown), the processor 150 may project acontent to a virtual flat screen based on the user voice, render thevirtual flat screen, and display the screen on the display 140.

FIG. 10 illustrates a flow chart for illustrating an operation of adisplay apparatus according to an embodiment of the disclosure.

The display apparatus 100 may generate a virtual flat screen 145 basedon the angle of the display 140 at operation S1010. Specifically, thedisplay apparatus 100 may set the coordinates of the four points 141,142, 143, 144 including the reference point 141 based on the angle ofthe display 140 and the horizontal and vertical lengths of the display140, and generate a virtual flat screen 145 based on the coordinates ofthe four points.

Next, the display apparatus 100 may correct the size of a content basedon the distance between the virtual flat screen 145 and the display 140at operation S1020. Specifically, the display apparatus 100 may correctthe size of a content based on information on correction coefficientswherein different correction coefficients are matched for each distancebetween the virtual flat screen 145 and the display 140.

Next, the display apparatus 100 may project the corrected content to avirtual space at operation S1030, and map the content projected to thevirtual space to the virtual flat screen 145 at operation S1040.Specifically, the display apparatus 100 may map the content projected tothe virtual space to the virtual flat screen 145 based on a curvedsurface-flat surface mapping algorithm. Here, the curved surface-flatsurface mapping algorithm is an algorithm for converting athree-dimensional curved surface to a two-dimensional flat surface, andit may be a parameterization polygonal mesh algorithm. However, this ismerely an embodiment, and the display apparatus 100 may convert athree-dimensional curved surface to a two-dimensional flat surfacethrough various algorithms such as a shape-preserving parameterizationalgorithm, etc.

Next, the display apparatus 100 may render the virtual flat screen 145to which the content is mapped at operation S1050, and display therendered virtual flat screen 145 at operation S1060.

Accordingly, a user may be provided with a content in an uncurved formeven in a state wherein the display 140 is folded.

FIG. 11 illustrates a flow chart for illustrating an operation of adisplay apparatus according to an embodiment of the disclosure.

The display apparatus 100 may identify a gaze vector based on thedirection of a user's gaze included in an image photographed through thecamera 120 at operation S1110. Specifically, the direction of the user'sgaze is a direction which a virtual line connecting from the centerpoint of the eyeball to the center point of the pupil is toward, and thedisplay apparatus 100 may calculate a point which proceeded toward theinside of the eyeball from the point wherein the user's pupil is locatedas much as the radius of the eyeball as the center point of the eyeball,and acquire a virtual line connecting from the center point of theeyeball to the center point of the user's pupil as the gaze vector ofthe user.

Next, the display apparatus 100 may divide the display 140 into a firstdisplay 710 and a second display 720 based on the line wherein thedisplay 140 is folded at operation S1120, and generate the virtual flatscreen 145 perpendicular to the gaze vector based on one point of thefirst display 710 and the gaze vector at operation 51130.

Specifically, the display apparatus 100 may identify a differencebetween an angle when the display 140 is in a flat state and an anglewhen the display 140 is in a folded state as the angle between thesecond display 720 and the virtual flat screen 730. As an example, ifthe angle of the display 140 is α, the display apparatus 100 mayidentify (180−α) degree as the angle θ between the second display 720and the virtual flat screen 730. However, this is merely an embodiment,and the display apparatus 100 may identify (the corrected referenceangle−α) degree as the angle θ between the second display 720 and thevirtual flat screen 730. Here, the corrected reference angle may be theangle when the display 140 is in a flat state.

Next, the display apparatus 100 may generate the virtual flat screen 730in a location wherein the second display 720 was rotated by the angle θbased on the folding line.

Next, the display apparatus 100 may project a content to be displayed onthe second display to the virtual flat screen based on the angle of thedisplay at operation S1140. That is, the display apparatus 100 mayidentify the angle θ between the second display 720 and the virtual flatscreen 730 based on the angle of the display, and project a content tobe displayed on the second display to the virtual flat screen based onthe rotation matrix.

Next, the display apparatus 100 may render the virtual flat screen towhich the content is projected at operation S1150, and display therendered virtual flat screen at operation S1160.

Accordingly, a user may be provided with a content in an uncurved formeven in a state wherein the display 140 is folded.

Meanwhile, the methods according to the various embodiments of thedisclosure as described above may be implemented in forms of software orapplications that can be installed on conventional display apparatuses.

Also, a non-transitory computer readable medium storing a program thatsequentially performs the control method of a display apparatusaccording to the disclosure may be provided.

A non-transitory computer readable medium refers to a medium that storesdata semi-permanently, and is readable by machines, but not a mediumthat stores data for a short moment such as a register, a cache, and amemory. Specifically, the aforementioned various applications orprograms may be provided while being stored in a non-transitory computerreadable medium such as a CD, a DVD, a hard disk, a blue-ray disk, aUSB, a memory card, a ROM and the like.

Although the present disclosure has been described with variousembodiments, various changes and modifications may be suggested to oneskilled in the art. It is intended that the present disclosure encompasssuch changes and modifications as fall within the scope of the appendedclaims.

What is claimed is:
 1. A display apparatus comprising: a sensor; adisplay; and a processor coupled to the sensor and the display, whereinthe processor is configured to: generate a virtual flat screen based onan angle of the display identified through the sensor, correct a size ofa content based on a distance between the virtual flat screen and thedisplay, project the corrected content to a virtual space, map thecontent projected to the virtual space to the virtual flat screen,render the virtual flat screen based on the mapped content, and displaythe rendered virtual flat screen on the display.
 2. The displayapparatus of claim 1, further comprising: a camera, and wherein theprocessor is further configured to: identify a user's pupil in an imagephotographed through the camera, identify a direction of a user's gazebased on a location of the identified user's pupil, rotate the virtualflat screen that the content is mapped in a direction perpendicular tothe direction of the user's gaze, render the rotated virtual flatscreen, and display the rendered rotated virtual flat screen on thedisplay.
 3. The display apparatus of claim 2, wherein the processor isfurther configured to: based on a gaze vector corresponding to thedirection of the user's gaze, identify a first vector perpendicular tothe gaze vector, based on the first vector and a second vectorcorresponding to the virtual flat screen, identify the angle between thefirst and second vectors, and rotate the virtual flat screen based onthe angle between the first and second vectors.
 4. The display apparatusof claim 3, wherein the processor is further configured to: based on theangle between the first and second vectors being an angle exceeding apredetermined angle in the direction wherein the user's pupil wasidentified based on the second vector, rotate the virtual flat screen inthe direction wherein the user's pupil was identified, and based on theangle between the first and second vectors being an angle exceeding apredetermined angle in an opposite direction to the direction whereinthe user's pupil was identified based on the second vector, rotate thevirtual flat screen in the opposition direction to the direction whereinthe user's pupil was identified.
 5. The display apparatus of claim 1,wherein the processor is further configured to: identify a rotatingdirection of the display based on a plurality of pulse signals ofdifferent phases received from the sensor, and identify the angle of thedisplay based on the rotating direction and a number of pulse signalsreceived from the sensor.
 6. The display apparatus of claim 1, whereinthe processor is further configured to: based on a coordinate of a firstpoint of the display, identify coordinates of second to fourth pointsusing the angle of the display, a horizontal length of the display, anda vertical length of the display, and generate the virtual flat screenbased on the coordinates of the first to fourth points.
 7. The displayapparatus of claim 1, wherein the processor is further configured to:identify a distance from the virtual flat screen to the display locatedin a vertical direction of the virtual flat screen, and correct the sizeof the content based on information on the identified distance andcorrection coefficients, wherein different correction coefficients arematched for each distance.
 8. The display apparatus of claim 7, whereinthe processor is further configured to: correct the content to be largeras a distance value from the virtual flat screen to the display becomeslarger.
 9. The display apparatus of claim 1, wherein the virtual spacethat the corrected content is projected is a three-dimensional spacehaving a form of a curved surface, and wherein the processor is furtherconfigured to: map the content projected to the three-dimensional spacehaving the form of a curved surface to the virtual flat screen having aform of a two-dimensional flat surface based on a curved surface-flatsurface mapping algorithm.
 10. A control method of a display apparatus,the method comprising: generating a virtual flat screen based on anangle of a display; correcting a size of a content based on a distancebetween the virtual flat screen and the display; projecting thecorrected content to a virtual space; mapping the content projected tothe virtual space to the virtual flat screen; rendering the virtual flatscreen that the content is mapped to; and displaying the renderedvirtual flat screen on the display.
 11. The control method of claim 10,further comprising: photographing a user, and wherein the displayingcomprises: identifying a user's pupil in the photograph, identifying adirection of a user's gaze based on a location of the identified user'spupil, rotating the virtual flat screen that the content is mapped to ina direction perpendicular to the direction of the user's gaze, andrendering the rotated virtual flat screen and displaying the renderedrotated virtual flat screen on the display.
 12. The control method ofclaim 11, wherein the rotating comprises: based on a gaze vectorcorresponding to the direction of the user's gaze, identifying a firstvector perpendicular to the gaze vector, and based on the first vectorand a second vector corresponding to the virtual flat screen,identifying the angle between the first and second vectors, and rotatingthe virtual flat screen based on the angle between the first and secondvectors.
 13. The control method of claim 10, wherein identifying theangle comprises: identifying a rotating direction of the display basedon a plurality of pulse signals of different phases received from asensor, and identifying the angle of the display based on the rotatingdirection and a number of pulse signals received from the sensor. 14.The control method of claim 10, wherein the generating comprises: basedon a coordinate of a first point of the display, identifying coordinatesof second to fourth points using the angle of the display, a horizontallength of the display, and a vertical length of the display, andgenerating the virtual flat screen based on the coordinates of the firstto fourth points.
 15. The control method of claim 10, wherein thecorrecting comprises: identifying a distance from the virtual flatscreen to the display located in a vertical direction of the virtualflat screen, and correcting the size of the content based on informationon the identified distance and correction coefficients, whereindifferent correction coefficients are matched for each distance.
 16. Thecontrol method of claim 15, wherein the correcting comprises: correctingthe content to be larger as a distance value from the virtual flatscreen to the display becomes larger.
 17. The control method of adisplay apparatus of claim 10, wherein the virtual space that thecorrected content is projected is a three-dimensional space having aform of a curved surface, and wherein the mapping comprises: mapping thecontent projected to the three-dimensional space having a form of acurved surface to the virtual flat screen having a form of atwo-dimensional flat surface based on a curved surface-flat surfacemapping algorithm.